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ABSTRACT 

For  a  certain  class  of  Iteration  schemes.  It  Is 
proved  that  the  method  of  successive  point  displacements 
converges  twice  as  fast  as  the  method  of  simultaneous 
point  displacements.   It  Is  also  proved  that  for  such 
schemes  the  rate  of  convergence  Is  unaltered  by  changes 
In  order  In  which  the  Iteration  Is  done* 
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THE  ITERATIVE  SOLUTION  OF  ELLIPTIC  DIFFERENCE  EQUATIONS 


I»  Introduction 

The  numerical  solution  of  partial  differential  equations 
is  usually  obtained  from  the  solution  of  the  set  of  linear 
equations  that  are  produced  by  the  replacement  of  the  partial 
derivatives  by  some  finite  difference  approximations.  When 
the  partial  differential  equation  is  of  the  elliptic  type, 
the  resulting  linear  equations  are  called  elliptic  difference 
equations.   Since  an  elliptic  equation  is  associated  with  a 
boundary-value  problem,  the  linear  equations  can  not  be  solved 
in  succession,  as  is  the  case  with  parabolic  or  hyperbolic 
partial  differential  equations,  but  must  all  be  solved  simul- 
taneously j  therefore,  the  problem  of  solving  a  set  of  elliptic 
difference  equations  is  a  special  case  of  the  general  problem 
of  solving  any  set  of  linear  equations.   However,  for  elliptic 
differential  equations  the  coefficient  matrix  contains  many 
zeros  (this  will  be  shown  later)  and  this  fact  suggests  the 
use  of  iterative  methods  for  solving  the  equations. 

This  note  studies  some  aspects  of  Iterative  methods.   It 
is  well -known '•■^■'  that  the  rate  of  convergence  of  an  iterative 
method  depends  upon  the  size  of  the  eigenvalue  of  maximum 
absolute  value  for  a  matrix  which  characterizes  the  Iteration 
method.   The  use  of  a  simple  lemma  about  determinants  (see 
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Leiuraa  3)  enables  us  to  discuss  the  eigenvalues  for  related 
matrices  and  thereby  to  discuss  also  the  rates  of  conver- 
gence of  different  iteration  methods. 

The  main  results  of  this  paper  are  two.  First,  a 
simplified  proof  of  a  theorem  (see  Theorem  II)  which  was 
first  proved  in  a  less  general  form  by  Yoimg'"   .   This 
theorem  states  that  in  a  three-block  scheme  the  method  of 
successive  displacements  is  twice  as  fast  as  the  method 
of  simultaneous  displacements.   Second,  it  is  proved  (see 
Theorem  III)  that  the  rate  of  convergence  for  the  method 
of  successive  displacements  in  a  three-block  scheme  is 
unaffected  by  the  order  in  which  the  successive  displace- 
ments are  done.  This  theorem  appears  in  a  slightly  dif- 

[3] 
ferent  form  in  an  unpublished  paper  by  Heller  -^  . 
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II.  Fundamentals 

Consider  a  system  of  p  linear  equations  in  p 
unknowns,  x^,  X2,  ...,  Xp.   The  iterative  method  for 
solving  these  equations  is  as  follows: 

Assume,  or  guess,  some  values  for  the  xinknowns.  We 
call  these  values  the  zeroth  approximation  to  the  correct 
values  of  the  unknowns.  Using  these  guessed  values,  we 
solve  the   p  equations  either  individually  in  some  order 
or  by  groups  to  obtain  a  new  set  of  approximate  values  for 
X  ,  ••.,  X  .  This  process  by  which  the  new  set  of  approxi- 
mations  was  obtained  from  the  original  set  is  repeated  to 
obtain  a  second  approximation.  Prom  the  second  approxi- 
mation we  obtain  a  third  approximation  and  so  on,  hoping 
that  the  approximations  converge  to  the  true  values  of  the 
unknowns . 

To  study  the  rate  of  convergence  of  such  an  iteration 
method,  we  introduce  matrices  and  vectors  in  a  p-dimenslonal 
space.  Let  x  be  a  p-dimensional  vector  whose  components 
are  the  correct  values  of  the  unknowns.  We  denote  by  x 
the  vector  whose  components  are  the  zeroth  approximations, 
that  is,  the  guessed  values,  of  the  unknowns,  and  by  x 
the  vector  whose  components  are  the  approximations  to  the 
\inknowns  which  are  obtained  after  n  repetitions  of  the 
iteration  process.   Since  the  next  approximation  x 

*It  is  possible  at  each  step  to  change  the  process  by  which 
the  new  approximations  are  obtained  but  in  this  paper  we 
shall  not  consider  this  possibility. 
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is  obtalnec  from  x     by  some  linear  process,  it  follows 
that  there  exists  a  matrix  K  and  a  vector  r  such  that 


(1)  x^^-'l)  =  K  x^^)  +  r   . 

Suppose  that  for  some  n  we  have  x    =  x  ;   then 
if  the  iteration  method  is  to  converge,  we  must  also  have 
x^^"^^^  =  X.   This  implies  that 


(2)  X  =  Kx  +  r. 


Consider  the  error  vector  z   '  =  x  -  x^   .   Prom 
(1)  and  (2)  It  is  clear  that 


(3)  z^^-'^^  =  Kz^")   . 

By  a  well-known  theorem  the  sequence   z     will  converge 
to  zero,,  no  matter  what  the  zeroth  error   z     is,  if  and 
only  ±£,   the  eigenvalues  of  K  are  all  less  than  one  in 
absolute  value.   We  define  the  spectral  norm  of  K  as  the 
maximum  modulus  of  the  eigenvalues  : ''  K.   ■>  taall  discuss 
the  form  of  K  and  its  eigenvalues  in  a  few  cases. 
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III.  The  Method  of  Simultaneous  Point  Displacements 

The  first  method  we  consider  Is  that  of  simultaneous 
point  displacements.   In  this  method  we  solve  one  particu- 
lar equation  for  the  first  unknown  x^  and  use  the  approxi- 
mate values  obtained  after  n  steps  for  the  other  unknowns 
Xp,  ...,  X  to  find  the   (n+l)-th  approximation  to  x-j^. 
To  clarify  this  procedure,  let  us  rearrange  the  system  of 
linear  equations  we  are  solving  so  that  the  equation  we 
solve  for  x,   is  the  first  equation,  for  y.^^     is  the  second, 
and  so  forth.   Suppose  the  first  equation  is 

^lA  -^  ^12^2  +  •••  -^  ^ip^p  =  ^1   • 
We  solve  for  x,   to  get 

x-L  =  -  (a^r.^2  +•••■*•  ^lp^p^/^11  ""  V^ll  • 

By  substituting  the   nth  approximate  values  in  the  right- 
hand  side  of  this  equation  we  obtain  the   (n+l)-th  approxi- 
mation to  X,  . 

Similarly,  we  solve  the  second  equation  in  the  rear- 
ranged system  of  linear  equations  for  x^  and  use  the 
approximate  values  obtained  after  n  steps  for  x^,  x^, 

X ,  X   to  obtain  the   (n+l)-th  approximation  to 

ij.'     '   n 

X  .   This  process  is  applied  to  each  unknown  in  turn, 
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always  using  the  n-th  approximate  values  of  all  the  other 
unknowns  to  obtain  the  (n+l)-th  approximation. 

Suppose  the  equations  are  arranged  in  the  order  in 
which  they  are  solved,  that  is,  arranged  so  that  the  k-th 
equation  is  solved  for  x,  .   If  the  linear  equations  in  this 
arrangement  are 


ik)  XI^lcj^j  =  ^k      k  =  1,  ...,  p  , 

•J   ^ 

or,  in  matrix  notation  with  the  obvious  meaning  for  the 
symbols. 

Ax  =  r   , 

then  the   (n+l)-th  approximation  is  obtained  from  the  nth 
by  the  following  formula: 

(5)  x^^-'l^  =  Kx^^)  +  r« 

where  K  is  the  matrix  whose  diagonal  elements  are  zero 
and  whose   (k,j)-th  element  is  "^-iri-Z^ir  ^^^  where 
r'  =  r^a,,  .   The  convergence  of  this  procedure  depends 
on  the  eigenvalues  of  K,   that  is,  on  the  roots  of  the 
equation 

(6)  det  IXI  -  K|  =  0   . 


-  9  - 


If  we  multiply  the  k-th  row  of  this  determinant  by  a,^,^ 
for  k  =  1,2,  ...,  p,   the  equation  will  be  multiplied 
by  a  constant  and  will  therefore  have  the  same  roots. 
However,  this  multiplication  changes  the  determinant  in 
(6)  to  the  following: 


(7) 


^^11 

^12 

•               •               •               • 

^21 

« 

Xa22 

• 

•  •               •               • 

•  •               •               • 

^1 

%2 

•               •               •               • 

a 


Ip 


2p 


Xa 


PP 


We  conclude  that  the  zeros  of  this  determinant  will  deter- 
mine whether  the  method  of  simultaneous  point  displacements 
converges  or  not. 

Suppose  that  instead  of  solving  for  the  \inknowns  in 
the  natural  order  x^,X2,  ...,  x  ,  we  decide  to  solve  for 

them  in  another  order,  say  in  the  order  x.  ,  x.  ,  ..,,  x.  , 

12        p 
that  is,  we  solve  for  x.  ,   in  the  first  equation,   x. 

^1  ^2 

in  the  second  equation,  and  so  on.   Does  this  change  the 

rate  of  convergence?  In  general,  it  will  and  the  rate  of 
convergence  will  be  determined  by  the  zeros  of  the  follow- 
ing determinant: 


(8) 


^^li^    '      ^li2 

,    ....,      a 

^2i^    '    ^^2i2 

,....,      a 

•         >       • 

,    •  •  •  •  , 

Vi    '      ^Pi2 

....      Xa 

li 


2i. 


pi. 
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x^ote  that  (8)  Is  obtained  from  (7)  by  changing  the  order 
of  the  columns  (without  the   X's)  from  1,  2,  ...,  p  to 
i,,  ip,  ...,  i    and  then  inserting  X's  along  the 
diagonals. 

However,  in  the  case  of  difference  equations  arising 
from  differential  equations  we  shall  see  that  it  is  both 
natural  and  logical  to  solve  the  1.   equation  for  x^^  . 
A  system  of  linear  equations  will  be  called  distinguished 
if  each  equation  of  the  system  distinguishes  the  unknown 
for  which  it  should  be  solved.  We  shall  prove  the  follow- 
ing lemma. 

Lemma  1.  When  the  method  of  simultaneous  displacements 
is  applied  to  a  distinguished  system  of  linear  equations, 
the  rate  of  convergence  will  be  the  same  no  matter  in  what 
order  the  equations  are  solved;  it  being  understood  that 
each  equation  is  solved  for  the  unknown  it  distinguishes. 

Let  us  so  number  the  equations  that  the  first  one 
distinguishes  x.  ,   the  second  Xp  and  so  on.   If  the 
equations  are  solved  in  the  order  1,  2,  .*.,  p,   the  zeros 
of  the  determinant  (7)  will  determine  the  rate  of  conver- 
gence of  the  method.   Suppose,  instead,  that  the  equations 

are  solved  in  the  order   1,,  1^,    .••,  1  .   This  means  re- 

1        d  P 

arranging  the  rows  of  (7)  (without  the   X's)  so  that  they 
are  in  the  order   1,,  ip,  •..,  i  .   But,  since  the   ii,~^ti 

-  11  - 


equation  Is  to  be  solved  for  x.  ,   the  colvunns  of  (7) 

must  also  be  arranged  in  the  order   1,,  i.^,    «..,  1  .   The 

final  result  is  that  both  the  columns  and  rows  of  (7)  have 

/I    2  ...  p  \ 
been  subjected  to  the  same  permutation  I  .    .     i  /  • 

1   2'**      p 
Under  this  transformation  diagonal  elements  of  (7) 

will  remain  on  the  diagonal.   For,  by  the  above  trans- 
forniation  the  j-th  row  goes  into  the  J'-th,  say,  and  the 
J-th  col-uran  goes  into  the  J'-th;   therefore  the  element 
a.,  goes  into  the   (j',j')   place  which  is  also  on  the 
diagonal.   Since  the  diagonal  elements  remain  on  the 
diagonal,  it  is  legitimate  to  consider  the  above  permu- 
tation applied  to  the  rows  and  columns  of  (7).   However 
a  permutation  of  both  the  rows  and  columns  of  a  matrix  may 
be  obtained  by  multiplying  the  matrix  in  front  by  a  permu- 
tation matrix  P  and  in  back  by  P"  .  Such  a  fore-and-aft 
multiplication  does  not  change  the  value  of  the  determinant 
of  the  matrix;  consequently,  the  determinant  with  rows  and 
columns   permuted   has  the  same  value  as  the  determinant 
of  (7).   This  proves  that  the  rate  of  convergence  for  dis- 
tinguished equations  is  not  changed  by  solving  the  equa- 
tions in  another  order. 
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IV.   The  Method  of  Block  Displacements 

Instead  of  solving  each  equation  for  one  unknown,  we 
sometimes  prefer  to  solve  a  subset  of  the  equations  for 
a  certain  subset  or  block  of  unknowns.   This  will  be  called 
the  method  of  block  displacements.   If  the  block  consists 
of  Just  one  unknown,  we  call  it  a  point  displacement.   Of 
course,  the  ideal  method  would  be  to  solve  the  complete 
set  of  equations  for  all  the  unknowns  at  once.   In  general, 
the  number  of  equations  is  so  large  that  this  method  is 
not  feasible.   However,  we  may  be  able  to  solve  for  2,  3, 
or  n  unknowns  at  a  time.   It  is  therefore  necessary  to 
investigate  the  rate  of  convergence  of  the  method  of  block 
displacements. 

'Ve  shall  illustrate  the  discussion  of  the  rate  of 
convergence  by  considering  the  case  where  the  block  consists 
of  Just  two  unknowns.   Going  back  to  the  equations  ([).)  we 
could  set  up  the  following  iteration  scheme : 

^lA      ^12^2      ^1  -%-  IJ  j 


(9) 


^2  A      ^22^2      ^2  -1-^2  j^  J 

(n+1)^     h+l)_  „  .  .   ^(n)_     (n)_  j-^        (n) 
33  3      3k  k      ~     3   31  1     32^2   4"  3J  J 


a-,-,x 


43  1+      kk  k  44II     422    -^i^-Jj 


-  13  - 


In  this  scheme  the  first  two  equations  would  be  solved 
for  X-, ,  Xp,   the  right-hand  side  being  considered  known, 
the  next  two  for  x^,  x,  ,   and  so  on. 
Introduce  the  matrices* 


(10)   N  = 


^11  ^12  °   ° 


^21  ^22  °   ° 


0   0   a33  a3^ 
0   0   a^3  a^ 


,    P  = 


y 


a^3  a.^^   ...  a^p 


23  ^2k  '"   ^2p 


.  a. 


a,  ,  a,  ^  0   0 


•••  \p 


then  (9)  may  be  written  as  follows: 


(11) 


Nx^^^l)  =  r  -  fe^^^ 


F'rom  this  It  is  easy  to  see  that 


Nz 


(n^l)  +  Pz^^)  =  0  ,   or,   z^^^l^  =  -  N-lpz^^^ 


where   z^^'  =  x-x^^';   consequently,  the  rate  of  convergence 
of  this  method  will  depend  on  the  eigenvalues  of  the  matrix 
-N"  P,   that  is,  on  the  zeros  of 


det  \X  +   N'-'-pj 


I  use  here  a  suggestion  of  Dr.  H.  B.  Keller. 
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or,  what  Is  equivalent,  the  zeros  of 

det  |XN  +  P|  =  det  N.det|X  +  N"-^P|   . 

For  the  case  we  are  considering  this  determinant  has  the 
following  form: 


Xa^^  Xa^2  *13  ^l^  •••  *lp 

^*21  ^^22     *23  *2l4.  *••  *2p 

*31  *32  ^^33  ^*34  •••  *3p 


detjXN  +  P|  = 


In  the  general  case  of  arbitrary  blocks,  the  rate  of 
convergence  will  also  depend  on  the  zeros  of  a  determinant 
|XN  +  P|  where  N  will  be  the  matrix  (similar  to  the 
left-hand  side  of  (10))  that  is  inverted  to  obtain  the 
new  approximation,  i»e.,  the   (n+l)-th,  whereas  P  is 
the  matrix  applied  to  the  previous  approximation,  i.e.  the 
n-th,  (similar  to  the  right-hand  side  of  (10)).   Note  that 
N  +  P  equals  the  original  matrix  of  the  system  (l^.).   From 
this  discussion  we  may  state  the  following  rule: 

The  rate  of  convergence  of  the  method  of  block  dis- 
placements will  depend  on  the  zeros  of  a  certain  determinant. 
This  determinant  is  the  same  as  the  original  determinant  of 
the  system  of  linear  equations  except  that  the  coefficients 
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of  each  block  of  unknowns  In  the  equations  that  are  used 
to  determine  them  are  multiplied  by  X. 

Consider  the  distinguished  set  of  equations 

^1 1^1  ~^1   *     ^~1»2,  •••,p   • 

If  ^^]^-   0  for  some  particular  values  of  1  and  k,  then 
when  the  1-th  equation  Is  used  to  find  a  new  approximation 
to  x^,     the  value  obtained  will  not  depend  on  the  value  of 
x^.  If  at  least  one  of  the  coefficients  a.,   or  a.  .   is 
not  zero,  we  shall  say  the  unknowns  x.  and  x,   are 
coupled.   If  both  a^^^^  and  a^^^^  are  zero,  we  shall  say- 
that  X.   and  X,   are  uncoupled*  Prom  the  previous  dis- 
cussion of  block  displacements  it  is  easy  to  show  the 
following: 

Lemma  2.   If  the  blocks  are  composed  of  uncoupled  un- 
knowns, then  the  method  of  block  displacements  is  Identical 
with  the  method  of  point  displacements* 
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V.   i^fethod  of  Successive  Displacements 

In  the  previous  methods  the  components  of  the   (n+l)-th 
approximation  were  found  by  always  using  the  values  of  the 
components  at  the  n-th  approximation.   In  the  present  sec- 
tion we  make  use  of  the  fact  that  in  practice  the  components 
of  the   (n+l)-th  approximation  are  not  all  found  simul- 
taneously but  Instead  are  actually  found  in  succession,  one 
after  the  other.   This  suggests  that  in  finding  the   (n+l)-th 
approximation  to  some  given  component  we  use  the  latest 
available  information,  that  is,  the  (n+l)-tb  approximation 
for  any  component  if  it  has  already  been  obtained.   For 
example,  we  may  use  the  following  Iteration  scheme  for  the 
equations  (ij.) : 


(12) 


.    • 


a  x^^-'l^^  r  -  '^a  x^^^ 
^lA      ^1  l-^lj'^j 

^22^2      ^2  ^21^1      •%-  2j^j 

33  3      3   31  1     ^j^2^2  t-  3j  j 

•       •     •         •        - 

PP   P  P    T—   PJ   J 


This  scheme  can  again  be  written  as 
Nx^^-'^L)  =  r  -  Px^^^ 

where 
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^11 


^21 


a 


22 


N  = 


^31 


^32   ^33 


and 


P  = 


a  T   a^5   a^^ 
pi    p2    p3 


0   a^2  a^2   '   ^Ip 


^23   •   ^2p 


the  omitted  elements  being  all  zero. 

We  call  the  Iteration  method  defined  by  (12)  Lhe 
method  of  successive  point  displacements.   Just  as  in  the 
preceding  section,  the  rate  of  convergence  of  this  nethod 
will  depend  on  the  zeros  of  the  determinant   |XN  +  Pi, 
that  is,  on  the  zeros  of 


(13) 


^^11  ^12  ^13 


Aa^-L   >^a22  ^2^ 


^IP 


a 


2p 


Xa^i   Xa-,o  ^a 


32  ^^33  •••  ^3p 


pi    p2   p3       PP 
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It  Is  also  possible  to  consider  a  method  of  successive 
block  displacements.   For  example,  in  (9)  the  iteration 
scheme  would  be  modified  as  follows: 


a  ^^""^^K   a  x^^-"^)  =  r  -  f"a  x^^^ 
^lA    ^  ^IZ'^Z  ^1  |-  Ij  J 


^21^ 


^^■^1)+  a  x^^^-l^-  r  -  f-a  x^^^ 
1      ^22^2      ^2  -^^2^^^ 


a.^xl^-^l^^-  a,,xf"-'l^=  r,-  a^.x;"  "'-  a..x 


33^3 


^3it."l^ 


(n+1)   „   ^(n+1) 
3"  '^31"!    "  ^32  2 


43  3       1+4  4       4   Ul  1      %-2  2 


-  3j  J 


and  so  on.   The  convergence  of  this  scheme  would  depend  on 
the  zeros  of  the  determinant 


(li|) 


^^11  ^^12   ^13   ^Ik     • 


Aa^,   Xa^-,   a^-,   &, 


^21 


'23    23    2k     • 


Xa^,   ^^-35  ^^'^'^  ^®"^]i   • 
^^la  ^^1^2  ^%3  ^^i|4  • 


Xa  T   Xa  -  Xa  ^   Xa  ,   . 

pi    p2    p3    m 


^IP 


.  a 


2p 


^3p 


%P 


. .  Xa 


PP 


A  study  of   X-determinants  such  as  (13)  and  {II4.)    show 
that  they  have  some  characteristics  in  common.   The  elements 
of  the  k-th  row  of  such  a  determinant  are 
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K    ^a,  . (  j=l,2, . . .  ,p)   where   (3.   can  be  either  zero  or  one. 
If  the  value  of   p,   is  zero  then  x.    should  be  used 

J  J 

in  the  iteration,  whereas  if  the  value  of   p,   is  one  then 
x.^     should  be  used.   Since  the   k-th  row  comes  from 
the   k-th  equation  and  since  this  equation  is  solved  for 
X,  ,   we  obtain  the  following  iteration  scheme: 


(15)         a^V-^1)  =  -5—a^.x(.^^Pj) 
kk  k        4^  kj  J   -J 


From  this  equation  we  deduce  the  following  character- 
istics of  all  X-determinants: 

1)  The  diagonal  elements  are  always  multiplied  by  X 

2)  All  those  elements  whose   (n+l)-th  approximate 
values  are  used  in  the  iteration  scheme  are  multiplied  by  X. 
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VI.   Elliptic  Difference  Equations 

V/hen  a  boundary-value  problem  for  an  elliptic  partial- 
differential  equation  is  to  be  solved  numerically,  the 
first  step  is  to  introduce  a  mesh,  usually  rectangular, 
into  the  region  on  which  the  boundary  values  are  given. 
The  next  step  is  to  replace  at  each  interior  mesh  point 
the  partial  derivatives  of  the  unknovm  function  by  some 
finite  difference  approximation  involving  the  values  of 
the  unlcnown  function  at  neighboring  mesh  points.   In  this 
way  the  partial  differential  equation  is  replaced  by  a 
set  of  algebraic  equations,  cne  equation  for  each  mesh 
point.   These  equations  are  called  elliptic  difference 
equations. 

For  example,  consider  the  problem  of  solving  the 
potential  equation 

Au  =  0 

in  the  rectangle   0<x<a,   0<y<t),   given  the  values 
of  u  on  the  boundary  of  the  rectangle.   The  lines 
X  =  ^^a,   (1=0,1,2,. .,,n+l)   and  ¥  =  -^^      (  j=0,l,2, . . .  ,ni-i; 
will  produce  a  mesh  on  the  rectangle.   Put 


^ij=  ^^Tlli  ^'   ^^) 


so  that  u . ,   is  the  value  of  the  unknown  function  at  a 
mesh  point.   Using  the  well-known  second  difference 
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approximation  to  the  second  derivative,  we  find  that  the 
potential  equation  is  replaced  by  the  following: 

u.  ,  .-2u.  4+u.,,  .   u.  .  T -2u,  .+u.  .,, 
(16)   -^^^^^ 1>.1   1^1  >.1  +  i».l-l ^1.1   ^».1+1  =  0  , 

2h^  Sh^ 

(i=l,2,...n  ;   j=l,2,,,.m)   •  • 

a  b 

Here   h,  =  -— rr  and  h^=  — rr  •   Note  that  u^  •»  ^  ,t  j» 
1  n+1        2  rti+1  0»J   n+l,j* 

u.  Q  and  u.   ,   are  known  from  the  given  boundary 
values;  consequently,  in  (16)  we  have  mn  equations  for 
the  mn  linknowns  u .  ,   (i=l,«..,n  ;   j=l,...,m). 

Note  that  equations  (16)  are  distinguished  because 
the  equation  obtained  at  any  mesh  point  distinguishes  the 
value  of  the  unknown  function  at  that  point.   It  is  theo- 
retically possible  to  use  the  equation  at  one  mesh  point 
to  solve  for  the  value  of  the  unlcnown  function  at  another 
mesh  point  but  in  practice  this  would  probably  give  a 
method  which,  if  it  converges  at  all,  converges  very  slowly, 

Hereafter,  we  shall  consider  only  elliptic  difference 
equations  and  therefore  only  distinguished  equations. 
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VII.  A  Lemma  on  Determinants 

We  shall  prove  a  lemma  about  determinants  that  will 
be  used  in  a  later  discussion  of  the  question  of  whether 
the  method  of  simultaneous  displacements  or  the  method  of 
successive  displacements  converges  more  rapidly  towards 
the  correct  solution.   Consider  the  square  matrix 


^11  ^12   •••  ^In 


(17) 


M  = 


^21  ^22   •••  ^2n 


•  •   • 


^nl  ^n2   •••  -^nn 

in  which  the  elements  A.  .  are  arbitrary  matrices  except 
that  the  diagonal  elements  A,^,  -^22*  •••»  ^         *r®  square 
matrices,  not  all  being  necessarily  of  the  same  order.   Let 

1^   (k=l,  2,  ...,  n) 

be  the  Identity  matrix  having  the  same  number  of  rows  and 
columns  as  the  square  matrix  A,  ,  ,   Put 


Q  = 


M-I. 


V-^I' 
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v"-^i„y 


where   |i.  is  any  non-zero  scalar.   Hote 


-1 


vr\^ 


-2t 


•  ^-(n-Dj 


,-1 


We  denote  the  matrix  QMQ    by  Mj   then  It  is  easy 
to  see  that 


'11 


^'\z 


(18)  M  = 


[xA 


21 


M-^A 


31 


22 

tiA 


13 


23 


32 


'33 


(n-l)„    nn-2.     ,,n-3. 


..  ,-(--2)a 


In 


2n 


3n 


..  A 


rm. 


J 


We  now  state  the  LenuTia  which  we  shall  use. 

Lemma  3.   If  M  and  M  are  the  matrices  defined  in  (17) 

and  (18).  then  det  M  =  det  M  . 

The  proof  follows  from  the  fact  that  the  determinant 
of  the  product  of  matrices  equals  the  product  of  the  deter- 
minants.  We  have 
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since 


det   M  =  (det  Q)(det  M)(det  Q.'-^) 


=  det   M 


(det  Q)(det  q,''^)   =  1      . 


-  25   - 


VIII.   Three  Block  Schemes  and  Young's  Property  A 

When  the  coefficient  matrix  for  a  set  of  elliptic 
difference  equations  is  considered,  it  is  found  that  a 
large  number  of  the  elements  are  zero.   The  reason  for 
this  is  that  the  value  of  the  unknown  function  at  any  point 
depends  only  upon  the  values  of  the  function  at  some  set  of 
neighboring  points.   In  other  words,  the  value  of  the  func- 
tion at  any  given  point  is  coupled  only  to  the  values  of 
the  function  at  some  neighboring  points.  For  example  in 
(16)  the  value  of  u.  .   is  coupled  only  to  the  values  of 
u._T  .,  u.^,  .,  u.  ,_,,  and  u.  .^,  .   It  Is  convenient  to 
say   in  this  case  that  the  point   (i,j)   is  coupled  to  the 
points   (i-l,j),  (i+l,j),  (i,j-l),  and  (i,j+l).  A  convenient 
way  of  representing  this  fact  is  by  the  following  pattern: 


Instead  of  considering  the  coupling  between  points,  we 
may  consider  the  coupling  between  blocks  of  points.   For 
example,  in  (16)  the  points  on  the  i-th  vertical  line  are 
coupled  only  to  the  points  on  the  (i-l)-th  line  and  the  (i+l)-th 
line.   We  shall  call  this  three  block  coupling  since  each  block 
(a  line)  is  coupled  only  to  the  block  preceding  it  and  the  block 
following  it.   We  represent  this  coupling  diagramatically  as 
follows : 
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X  X  X  X  X  X  X 

0  0  0  0  0  0  0 

X  X  X  X  X  X  X 

An  iteration  scheme  which  uses  block  displacements 
and  In  which  each  block  is  coupled  only  to  the  block  before 
and  the  block  after  is  called  a  three-block  scheme.   For 
example,  in  (16)  an  iteration  scheme  based  on  vertical  or 
horizontal  line  displacements,  either  simultaneous  or  suc- 
cessive, will  be  a  three-block  scheme.   Notice  also  that 
the  method  of  simultaneous  point  displacements  in  (16)  can 
be  considered  as  a  three-block  scheme  if  the  lines  parallel 
to  a  diagonal  of  the  rectangle  are  taken  as  blocks.   This 
result  follows  from  these  two  facts:   First,  points  on  the 
same  "diagonal"  are  vincoupledj  consequently,  the  method  of 
"diagonal"  block  displacements  is  the  same  as  the  method 
of  point  displacements.   Second,  a  diagonal  contains  all 
points   (l,j)   such  that   i+j  =  constant  =  k,   say;  from 
(16)  it  follows  that  the  diagonal  defined  by  the  constant 
k   is  coupled  only  to  the  diagonals  defined  by  k-1   and 
k+1.   These  facts  also  follow  from  the  geometrical  diagram 
given  before.   If  we  combine  the  pattern  for  points  on 
the  same  diagonal,  we  get: 

0  X 
X  0  X 
X  0  X 
X  0  X 
X  0  X 
X  0 
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This  pattern  immediately  indicates  the  presence  of  a  three- 
block  scheme. 

Three-block  schemes  are  closely  related  to  matrices 

[p] 
which  possess  Young's  Property  A    .  A  matrix  with  elements 

a. .  (i=l,...p;  j=l,...p)   is  said  to  have  Property  A  if 

there  exists  a  decomposition  of  the  integers   l,..,,p  Into 

two  non-overlapping  non-ompty  subsets   S  and  T  such  that 

if  a.  .  4  0  then  either  i=j  or  one  of  the  integers  1 

and  j  belongs  to  S  and  the  other  belongs  to  T.   We 

shall  prove  the  following  theorem. 

Theorem  I.   If  a  matrix  M  possesses  Property  A  then 
the  method  of  simultaneous  displacements  for  M  Is  a  three- 
block  scheme.   Conversely,  any  matrix  for  which  a  three- 
block  scheme  exists  possesses  Property  A. 

Proof.   Assume   M  possesses  property  A.   We  shall  say 
that  X.   belongs  to   S  or  T  if  1  belongs  to   S  or  to 
T,  respectively.   If   1  and   1'   are  non-equal  integers 
which  are  both  in  S  or  in  T,   then  a.. ,=  a, ,.=  0  and 
consequently  x^   is  vmcoupled  with  x. , .   If  all  the  unknowns 
belonging  to   S  are  considered  as  one  block  and  all  the 
unknowns  belonging  to  T  are  considered  as  another  block, 
Lrraa  2  implies  that  the  method  of  block  displacements  for 
these  blocks  is  the  same  as  the  method  of  point  displace- 
ments.  Since  there  are  only  two  blocks,  it  is  obviously  a 
special  case  of  a  three-block  scheme  and  the  first  state- 
ment of  the  theorem  is  proved. 
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Suppose  now  that  a  three-block  scheme  exists  for  the 

matrix  M.   Let  the  blocks  of  unknowns  be  denoted  by  A^, 

A^,  ....  A  .   V;e  shall  define  S  and  T  by  specifying 
2     '   m 

the  unknowns  which  belong  to  them.   The  unknown  x^  belongs 
to  S  or  T  accordingly  as  x^  is  in  a  block  having  an 
odd  or  even  subscript.   From  the  three-block  property  an 
unknown  in  a  block  with  an  odd  (even)  subscript  is  coupled 
only  to  the  unknowns  in  a  block  with  an  even  (odd)  sub- 
script.  This  means  that  if  a^  .^  0,   either  i=j  or  x^^ 
and  X.   cannot  both  belong  to  either   S  or  T.   This  is 

%J 

Property  A;  therefore  the  theorem  is  proved. 
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IX«   Simultaneous  versus  Successive  Displacements 

In  the  method  of  simultaneous  displacements  the   (ni-l)-th 
approximation  is  foiind  by  using  only  the  values  given  pre- 
viously at  the  n-th  approximation  whereas  in  the  method 
of  successive  displacements  the   {n+l)-th  approximation  Is 
found  by  using  the  most  recent  values,  namely,  the  values 
at  the   (n+l)-th  approximation  if  they  are  available,  other- 
wise, the  values  at  the   n-th  approximation.   It  seems 
plausible  to  expect  the  method  of  successive  displacements 
to  converge  more  rapidly  tlian  the  jTiethod  of  simultaneous 
displacements.   The  following  theorem  shows  that,  for  three- 
block  schemes,  this  expectation  is  justified. 

Theorem  II.  In  a  three-block  scheme  the  spectral  norm 
for  the  method  of  successive  displacements  is  the  square  of 
the  spectral  norm  for  the  method  of  simultaneous  displacements. 

This,  of  course,  implies  that  when  the  method  of  simul- 
taneous displacements  converges,  then  the  method  of  succesive 
displacements  converges  more  rapidly. 

The  proof  follows  from  Lemma  3  on  determinants.   In  a 
three-block  scheme  the  determinantal  equation  for  the  method 
of  simultaneous  displacements  will  be 


det  M  ^ 
a 


XA. 


B, 


XA, 


B, 


XA 


3 


B. 


m 


XA 


m 


=  f(X)  =  0 
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where     A^ ,   A~,    ...,   A        are    square   matrices.      For  the   method 
Id  HI 

of  successive   dlaplaceraents   the   deterrainantal  equation   is 


det   M^  = 


XA^     Bj_ 


XCp   XAp      Bp 


XC-    XA-      B^ 


XC      XA 
m       m 


=   0 


1/2 
Put     ^L  =  X  '     ,      then  by   the   Lemma  on  determinants 


det  M^=  det  K^= 


(19) 


=  .p/2 


=   X 


XA, 

x^/\ 

x^/% 

XAg 

x^/\ 

X^/^Cj 

XA3 

x1/2b 

.^/\     B, 


X^/^A2      B2 


X-'-Z^A^     B- 


Suppose  the  spectral  norm  for  M   is 


X^/^C   XA 
m    m 


C    x1/2a 
m       ra 


=xP/2f(xl/2)=o 


s  =  X  ! 
'  a 
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then  the  spectral  norm  for  M.   is 


t  =  \X^ 


where  by  (19) 


a   1 


2 

Prom  this,  it  follows  that   t  =  s  ,   which  was  to  be  proved, 
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X.   Different  Orderlngs 

We  have  shown  that  for  the  method  of  simultaneous 
displacements  the  order  in  which  \ie    solved  for  the  unknowns 
has  no  effect  on  the  rate  of  convergence.   However,  for  the 
method  of  successive  displacements  different  orderings  of 
the  unknowns  will  give  different  rates  of  convergence.   In 
this  section  we  shall  consider  some  cases  in  which  varying 
the-  order  in  which  we  solve  for  the  unknovrns  will  not  affect 
the  rate  of  convergence. 

Let  us  review  the  method  of  successive  block  displace- 
ments where  the  blocks  are  labeled  1,  2,    «..,  n  accordinr 
to  the  order  in  which  they  are  solved,  (i^ote  that  the  blocks 
may  reduce  to  points.)   The  method  is  as  follows: 

An  initial  guess  for  the  values  of  all  the  unknowns 
in  the  blocks  is  roade .   With  the  help  of  this  guess,  the 
value  of  block  1,  that  is,  the  values  of  the  unknovms  in 
block  1,  is  found;  the  value  of  block  2  is  found  by  the  use 
of  the  value  of  block  1  and  the  remaining  guessed  values; 
the  value  of  block  3  is  obtained  by  using  block  2,  block  1 
and  the  remaining  guessed  values;  and  so  on.   If   G  denotes 
the  guessed  values,  we  may  represent  the  order  in  which 
successive  blocks  are  found  by  the  following  sequence: 

(20)     G,  1,  2,  3,  ...,  n,  1,  2,  3,  ...»  n*  1>  2,  3,  ••• 

This  sequence  indicates  that  to  find  the  value  of  any  block 
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we  use  the  values  of  all  the  blocks  preceding  it  and  the 
guessed  values  of  all  the  blocks  which  have  not  been  found 
up  to  that  point.   The  repetition  of  the  nu:iibers   1,  2,  ...,  n 
indicates  that  after  wo  have  found  the  value  of  all  the 
blocks  from  1   to   n,   we  start  over  again  with  block  1  and 
repeat  the  process  indefinitely. 

1/e  prove  the  follov;ing  lemma: 

LenLiia  It..   If.  in  the  method  of  successive  block  d1s- 
placevients.  the  method  begins  with  an  arbitrary  numbered 
block  instead  oi  with  block  1  but  the  succession  of  blocks 
is  .just  a  cyclical  rearranrement  of  the  number   1,  2,  ...»  n, 
then  the  rate  of  convergence  is  unaltered. 

We  present  the  proof  for  the  case  where  the  blocks 
are  evaluated  in  the  order  2,  3»  ....f  n,  1  ;  the 
proof  for  any  other  cyclical  rearrangement  would  be  the 
same.  In  this  case  the  method  starts  with  a  guess  G,  j 
then  the  value  of  block  2  is  found  by  using  the  guessed 
values;  then  the  value  of  block  3  is  found  by  using  the 
value  of  block  2  and  the  guess;  and  so  on.  The  method  may 
be  represented  by  the  following  sequence: 

(21)         G^  ,    2,  3»  ••»>    ^t    Ij  2,  •..,  n,  1,  2,  ... 

Comparing  this  sequence  with   (20),   we  see  that  they  would 
be  exactly  the  same  if   G,   was  equal  to   G   in   (20)   modi- 
fied by  the  value  found  for  block  1.   Since   G  and   G,   are 
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completely  arbitrary  guesses,  it  is  always  possible,  given 
G,   to  find  G^   and  conversely,  given  G^ ,   to  find   G 
such  that  the  methods  represented  by   (20)   and   (21)   give 
exactly  the  same  values  for  the  blocks.   This  proves  the 
lemma. 

The  question  of  what  effect  an  arbitrary  permutation 
of  the  ordering  has  on  the  rate  of  convergence,  in  general, 
is  still  open.   However,  for  a  three-block  scheme  the  fol- 
lowing theorem  provides  the  answer. 

Theorem  IIl"^   In  a  three -block  scheme  the  rate  of  con- 
vergence of  the  method  of  successive  displacei'ients  is  the 
same  for  all  possible  orderings  of  the  bloc.is. 

Since  all  possible  permutations  of  the  ordering  can  be 
obtained  as  a  product  of  transpositions,  it  is  sufficient 
to  consider  the  effect  on  the  rate  of  convergence  of  a 
transposition  of  two  blocks.   Suppose,  then,  that  the  blocks 
are  solved  in  the  order  1,  3>  2,  [j.,  ...,  n.   The  proof  for 
any  other  transposition  will  be  similar.   The  sequence  for 
the  "transposed"  method  is 

{2Z)  G^,  1,  3,  2,  [j.,  ...,  n,  1,  3,  2,  ij.,  ...,  n,  1,  3,  2,  i;,  ... 

Wow,  in  a  three-block  scheme  any  block  is  coupled  only  to  the 
block  before  it,  with  nvunber  one  less,  and  the  block  after 

''This  result  is  in  an  unpublished  paper  by  Dr.  J.  Heller. 
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it  with  number  one  more.   The  exceptions  to  this  are  blocks 
1  and  n  which  are  coupled  only  to  blocks  2  and  n-1  respec- 
tively.  In  particular,  block  3  is  coupled  only  to  blocks  2 
and  4.   This  implies  that  a  knowledge  of  the  value  of  block 
1  has  no  effect  on  the  value  of  block  3;  therefore,  we  may 
solve  for  block  3  before  solving  for  block  1.   This  means 
that  the  method  illustrated  by  (22)   is  the  same  as  the 
method  illustrated  by  this  sequence: 

(23)    Gg*  3,  1,  2,  4,...,  n,  3,  1,  2,  l\.,    ...,  n,  3,  1,  2,  I^, 

The  argiiment  we  have  used  for  block  3  "lay  be  applied  to 
blocks  [|.,  ...,  n.   These  blocks  are  not  coupled  to  blocks  1 
and  2;  consequently,  a  knowledge  of  the  values  of  blocks  1 
and  2  has  no  effect  on  the  values  of  blocks  I|.,  «..,  n  and 
conversely;  therefore,  we  may  solve  for  blocks  Lj.,  «.,  n 
before  solving  for  blocks  1,  2.   This  means  that  the  method 
illustrated  by   (23)   is  the  same  as  the  method  illustrated 
by  the  following  sequence : 

(2ij.)    G^,  3,  k>    •••*  n,  1,  2,  3,  ky    •••*  n,  1,  2,  3,  k>    ••• 

However,  the  method  illustrated  by  {21^)    involves  only  a 
cyclical  rearrangement  of  the  ordering.   By  Lemma  I4.  the  rate 
of  conver^^ence  for  (2i|)  is  the  same  as  for  (20).   Since  (22) 
and  (2I4.)  are  illustrations  of  essentially  the  same  method, 
this  proves  the  theorem. 
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